wealth 

# % 

J? UCLA o. 

~S 0 enter „ 


& 


% 


°ndm\ 


/ 


A Center Brief * 


Implementation Science and Innovative Transformation of 

Schools and Communities 

(March, 2013) 


Abstract 

Increasing attention to Implementation Research and the Implementation 
Problem has given rise to confusion about matters such as the role of 
empirically-supported practices, fidelity of implementation, and monitoring fidelity 
of implementation. To clarify the matters, we approach these topics from the 
broad perspective of efforts to transform schools and their relationship to the 
surrounding community and with reference to the literature on diffusion of 
innovations and enabling major systemic changes. Such a perspective points to 
the need to expand implementation research and practice in ways that focus on 
the complexities of (a) facilitating essential systemic changes for implementing 
a comprehensive approach at specific sites, (b) replicating the approach across 
a school district, and (c) sustaining and evolving what has been implemented. 
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Implementation Science and Innovative Transformation of 
Schools and Communities 

W ith the increasing attention to Implementation Research and the Implementation 
Problem, some confusion has arisen, especially with reference to the emphasis on 
empirically-supported practices, fidelity of implementation, and monitoring fidelity 
of implementation. The intent here is to help clarify these matters. We do so from the broad 
perspective of efforts to transform schools and how they relate to the surrounding 
community. 

Clearly, the state of the art related to the above matters is still quite narrow and limited in its 
nature and scope. As the National Implementation Research Network (NIRN) has stressed: 

"... very little is known about the processes required to effectively 
implement evidence-based programs on a national scale. Research to 
support the implementation activities that are being used is even scarcer." 

The field is at a stage where many of us are just becoming more knowledgeable about the 
complexities and strategies related to diffusion of innovations, enabling major systemic 
changes, and developing a sophisticated understanding of the role of empirically-based 
practices (Flaspohler, Lesesne, Puddy, Smith, & Wandersman, 2012; Norcross, Beutler, & 
Levant, 2005; Rabin & Brownson, 2012; Rogers, 2003). 

At its roots, the implementation problem refers to efforts to ensure a practice is carried out. 
A special case of the implementation problem involves taking prototypes that are found 
efficacious under highly controlled conditions and moving them into the real world. Much 
of the recent research on implementation has focused on this special case. This has resulted 
in the tendency for some researchers, practitioners, and policy makers to skip by 
fundamental considerations that are at the core of efforts to transform public education, 
public health, and community development. The deficiencies become apparent when 
implementation efforts are analyzed in terms of the complexities related to (1) diffusing 
comprehensive and multifaceted innovations and (2) doing so in the context of organized 
settings with well-established institutional cultures and infrastructures that must change if 
effective widespread application is to take place (Centers for Disease Control and 
Prevention, 2012; Institute of Education Sciences, 2013). 

The following discussion approaches the implementation problem from the vantage point 
of the growing bodies of literature on diffusion of innovations and systemic change. 
Moreover, the focus is on the complexities of transforming how a school district and 
community implement an innovative, comprehensive strategy designed to unify related 
interventions into and cohesive system for addressing the many barriers interfering with 
learning and teaching (as contrasted to implementing a discrete service or program to address 
a narrowly defined problem). 

From this perspective, the implementation problem is framed as a process of diffusing a 
comprehensive innovative strategy through major systemic changes. Such a perspective 
focuses research and practice on the complexities of (a) facilitating essential systemic 
changes for implementing an appropriate and effective adoption/adaptation of a 
comprehensive prototype at specific sites, (b) replication-to-scale across a school district, 
and (c) sustaining and evolving what has been implemented. 
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Understanding 
the Limitations 
of Empirically- 
Supported 
Practices 


Simply because an 
intervention has 
produced positive 
findings is not a 
sufficient rationale 
for adoption by 
schools and 
communities. 


Empirically-supported practices also are referred to as 
evidence-based and science-based practices. The terms refer to 
any intervention that has been identified as having research data 
generated using methods that meet scientific standards and that 
demonstrate a level of efficacy deemed worthy of application in 
natural settings. Ideally, such application is to be followed by an 
evaluation of effectiveness and, as appropriate, replication on a 
large scale (Institute of Education Sciences, 2013; Kratochwill & 
Shernoff, 2004; La Roche & Christopher, 2009). 

A subgroup of empirically-supported practices, referred to as 
empirically-supported or evidence-based treatments, focuses on 
differentially diagnosed illnesses and disorders. According to the 
American Psychological Association, the designation of 
evidence-based treatment should be reserved for those 
interventions that have been tested in more than one scientifically 
rigorous study (either multiple case studies or randomized control 
trials) and consistently have been found to work better than a 
placebo or no treatment (American Psychological Association, 
2012). Such treatments usually are applied using a manual and are 
time-limited. 

Most evidence-based practices are discrete interventions designed 
to meet narrow-band goals. A few are complex sets of 
interventions intended to meet several related goals, and these 
usually are referred to as programs. 

An empirically-supported practice may or may not be a best 
practice. A best practice is one that decision makers view as the 
most productive (e.g., cost-effective) approach that has been 
identified for achieving some desired results. Identifying such a 
practice usually involves extrapolating from available research 
and drawing on the experience of those with special expertise. 
When formal research is inadequate for extrapolation, the 
empirical support for best practices usually stems from 
experiential expertise. 

Empirically-supported interventions all can be seen as helping to 
advance research and theory about intervention, and efforts to 
implement such practices can help inform implementation science. 
However, simply because an intervention has produced some 
positive findings is not a sufficient rationale for adoption by 
schools and communities. Similarly, research on implementation 
of such practices is too narrow in nature and scope to provide 
detailed guidance for making major systemic changes in the 
context of schools and communities. 
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Some Implications for Schools and Community 

From a systemic, public education, public health, and community 
development perspective, a major concern is that the science-base for many 
practices is limited to discrete services and programs, has been developed 
under highly controlled conditions, and for a relatively brief period of time. 
There is no guarantee that such practices will produce the same outcomes 
when applied widely or that they can be sustained over time. In effect, until 
researchers demonstrate that a prototype is effective under “real world” 
conditions, it is a promising not a proven practice. And, even then it must be 
determined whether it is a best practice. 

Findings from highly controlled studies are referred to as data on efficacy, 
findings from studies conducted under common conditions of daily practice 
are designated as data on effectiveness. In both instances, concern about 
generalizability arises when studies have not included samples representing 
major subgroups with whom the practice is to be used. Another major 
concern is that certain interventions increasingly are officially prescribed and 
others are proscribed by policy makers and funders, and only those 
practitioners who adhere to official lists are sanctioned and rewarded. This 
is a particular concern in sectors where individual needs come into conflict 
with powerful social, political, and economic forces. 

Moreover, introduction of any new practice into an organization raises 
questions about where the proposed practice fits and how best to weave it 
into a comprehensive continuum of interventions. For example, with respect 
to children and adolescents, most schools and communities offer a range of 
programs and services oriented to youngsters’ needs and problems. 
Introducing something new can add to the widespread problems of 
fragmented and marginalized approaches and counterproductive competition 
for resources. 

Of more fundamental importance, the nature and scope of currently available 
empirically-supported practices are insufficient for accomplishing the type of 
innovative transformation needed to ensure equity of opportunity for all 

(Cont.) 


4 


( Cont.) 


students to succeed at school and beyond. This is illustrated by the U.S. 
Department of Education’s What Works Clearinghouse. The Clearinghouse 
seeks out all research studies on discrete interventions, reviews each against 
evidence standards, and summarizes the findings of those that meet their 
standards in Intervention Reports. At the same time, they compile Practice 
Guides to aid educators in dealing with general classroom and school 
concerns, These guides are “based on reviews of research, the experiences 
of practitioners, and the expert opinions of a panel of nationally recognized 
experts.” It is noteworthy that the guides for turning around low performing 
schools all have minimal evidence. 

Given all this, as schools and communities seek to add best practices, they 
need to adopt an overall intervention framework. And if the aim is innovative 
transformation of current systems, the framework must provide a big picture 
vision of both a full continuum and general arenas of intervention. In contrast 
to discussing discrete practices in narrow ways and often out of context, such 
a framework provides a big picture vision that guides analyses about critical 
gaps and decisions about priorities and emphasizes the weaving together of 
school and community resources. As an example, see our work related to 
framing and operationalizing a unifying and comprehensive system of 
interventions for schools and communities focusing on addressing barriers 
to learning and teaching and re-engaging disconnected students (e.g., see 
Adelman & Taylor, 2006, 2010). 

Empirical research on the entire package of interventions outlined by any 
comprehensive intervention framework is difficult under the best of 
circumstances, and a continued overemphasis on discrete interventions 
makes it unlikely that the right circumstances will forthcoming in the near 
future. 
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Understanding the 
Limitations of 
Narrowly Focusing 
on Fidelity of 
Implementation 


In almost all 
instances where 
a prototype 
design is to be 
implemented in 
schools, some 
adaptation is 
required. 


Many researchers interested in having schools adopt 
evidence-based practices have discussed the implementation 
problem mainly in terms of enhancing fidelity of implementation 
Carroll, Patterson, Wood, Booth, Rick, & Balain, 2007; O’Donnell, 

2008) . From this perspective, fidelity is discussed as involving 
implementation of a prototype in accordance with its 
conceptualization and the specifics of its design. Sometimes the 
emphasis on fidelity is broadened to encompass implementation of 
at least a prototype's essential principles, core components, or 
"active agents" (Beutler & Johannsen, 2005; Chorpita & Daleiden, 

2009) . 

Formulating the implementation problem in terms mainly of fidelity 
ignores the many fundamental complexities related to diffusing 
comprehensive innovations that require fundamental systemic 
changes (e.g., such as those that are essential in transforming how 
schools and communities weave together resources to address 
barriers to learning and teaching and re-engage disconnected 
students). For example, with reference to schools and their 
relationship to the surrounding community, concerns arise about 
valid adaptation to ensure successful systemic change. Back in 
2004, Castro, Barrera Jr. & Martinez Jr (2004) described the matter 
as a dynamic tension regarding two imperatives: (a) fidelity of 
implementation and (b) program adaptation - the modification of 
program content to accommodate the needs of a specific consumer 
group. As Richard Price (1997) states the matter: 

"Effective implementation depends not on 
exclusive and narrow adherence to researcher 
definitions of fidelity, but on mutual adaptation 
between the efficacious program features and needs 
and competencies of the host organization." 

From this perspective, the implementation problem expands to 
include facilitating effective adoption/adaptation of a prototype at 
a particular site. 

Moreover, when implementation involves major changes in an 
institutional setting (such as a school), it is essential to account for 
the reality that such settings have well-ingrained policies, culture, 
and infrastructures. Stated differently, it is unrealistic and 
inappropriate to expect established organizations to make major 
systemic changes without adapting prototypes so that they fit or 
reworking policies, culture, and infrastructures to improve the fit. 
Related to this, it is important to consider the renewed and growing 
interest in replacing prescriptive practices with approaches that 
emphasize essential principles and critical elements of what makes 
an approach effective. 
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Good ideas and 
missionary zeal are 
sometimes enough to 
change the thinking of 
individuals; they are 
rarely, if ever, effective in 
changing complicated 
organizations (like the 
school) with traditions, 
dynamics, and goals of 
their own. 

Seymour Sarason 


Then come the added complexities of system-wide replication- 
to-scale, sustainability, and creative renewal. Such concerns, of 
course, are compounded when introducing comprehensive 
initiatives (as contrasted to specific, narrowly-focused programs 
and services). And are further complication when interventions 
are novel. 

All this requires processes and mechanisms conceived in terms 
of implementation, systemic change, transformation, replication 
to scale, sustainability, and creative renewal. The tasks related 
to each of these matters usually require policy revisions and 
significant changes in existing operational infrastructures. The 
work also calls for the skills of a well-trained change agent and 
well-designed capacity building (e.g., technical assistance, 
coaching, stakeholder development). Also important is an 
appreciation of psychological considerations that influence 
effectiveness (e.g., motivation, interpersonal dynamics). 


At the same time that so much emphasis has been placed on 
fidelity, a growing body of literature is exploring the problem of 
applying science to advance practice (e.g., Centers for Disease 
Control and Prevention, 2012; Flaspohler, Lesesne, Puddy, 
Smith, & Wandersman, 2012; Greenhalgh, Robert, Macfarlane, 
Bate, & Kyriakidou, 2004; Institute of Education Sciences, 2013; 
Layde, Christiansen, Peterson, Guse, Maurana, & Brandenburg, 
2012). This work stresses the complexities of the implementation 
problem and is offering broad frameworks to guide 
implementation research. 


Discussion of Fidelity Often Diverts Attention from 
More Fundamental Concerns 

Not only does the narrow focus on fidelity often limit implementation research, 
the discussion of fidelity tends to divert attention from the more fundamental 
question of whether introduction of another discrete practice is premature. For 
example, schools have many needs and limited budgets. In general, school and 
related community decisions about what interventions to pursue should be 
based on what is likely to account for major progress toward ensuring the 
greatest good for the many and not just another specific practice for a relatively 
small number of students. The complex problem for school decision makers, 
of course, is how to address the needs of all students in ways that support 
commitment to social justice and enhancing equity. 
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Formative 

Evaluation: 

Beyond 

Monitoring 

Fidelity 


Summative 
evaluation is 
done only after 
effective 
implementation. 


Clearly, the implementation problem is highly complex, especially 
when viewed from the intent to apply comprehensive and 
innovative approaches on a large-scale (e.g., the type of 
comprehensive and cohesive set of transformative interventions 
needed to ensure equity of opportunity for all students to succeed 
at school and beyond). Clearly, the process goes well beyond the 
matter of fidelity. As a result, monitoring fidelity becomes just 
one facet of comprehensive formative evaluation and rapid 
problem-solving (Rossi, Lipsey, & Freeman, 2004). 

Effective formative evaluation grows out of strategic planning 
designed to ensure that the vision for innovative transformation is 
effectively and efficiently implemented. The plan spells out an 
answer to: How do we get therefrom here? 

Given that the work involves major systemic changes, the focus 
is on four major phases (Adelman & Taylor, 2007): 

(1) Creating Readiness and Commitment - enhancing the 
climate/culture/conditions for innovative systemic 
change 

(2) Start-up and phase in - initial implementation (which 
includes decision making to adopt/adapt and phase-in an 
innovation with well-designed infrastructure and 
capacity building) 

(3) Sustaining, evolving, and enhancing outcomes - 
ensuring institutionalization, maintenance, momentum, 
replication to scale, and progress 

(4) Ongoing evolution and creative renewal. 


For each phase, given prevailing antecedent conditions, strategic planning clarifies the 
actions to be taken (e.g., the who, what, when, and how) with respect to such matters as: 

• leadership and staffing for overseeing, steering, and implementing the strategic plan, 

• capacity building, 

• establishing ways to monitor all facets of implementation (e.g., benchmarks and 
processes for assessing progress), 

• formulating ways to evaluate effective implementation (e.g., indicators, standards, 
deliverables, immediate and longer-term impact in terms of costs-benefits). 

Given the presence of relevant antecedent conditions, key formative evaluation concerns 
include whether intended processes actually transpire and whether potent unintended 
processes occur. Findings related to such matters are basic in deciding what implementation 
changes are needed. 
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Examples of Formative and Summative Evaluation Aids 

An example of benchmark indicators from our work is included as Appendix A. 

Also see: 

>Monitoring Progress in Developing a Comprehensive System to Address 
Barriers to Learning and Teaching - Quick Overview Guide for Self-Evaluation 
http://smhp.psych.ucla.edu/pdfdocs/studentsupport/toolkit/selvevaltool.pdf 

Examples of standards and related indicators are in 

>Common Core Standards for a Learning Supports Component 
http://smhp.psych.ucla.edu/pdfdocs/commcore.pdf 

With respect to impact/outcome data related to addressing barriers to learning and 
teaching, see Appendix B. Also see: 

> About Short-term Outcome Indicators for School Use and the Need for an 
Expanded Policy Framework 

http://smhp.psych.ucla.edu/pdfdocs/outind.pdf 

>Expanding the Accountability Framework for Schools 
http://smhp.psych.ucla.edu/pdfdocs/account.pdf 


Concluding Comments 

The intent of this brief has been to underscore that efforts to transform schools and 
their relationship to the surrounding community require dynamic, comprehensive 
transactional approaches that go beyond the available science-base. Analyses of the 
literature make it clear that restricting the focus to practices that have undergone 
rigorous study is too limiting. Prototypes for comprehensive and innovative strategies 
are yet to be researched. This also the situation with respect to implementing such 
approaches at specific sites, replicating them to scale, and sustaining and evolving 
them over time. 

So, while policy makers continue to push for applying science to help make 
fundamental improvements to schools and communities, they also must accept that it 
will be some time before the science-base is sufficient to prescribe and proscribe the 
what and the how. The desire of decision makers to apply science needs to be balanced 
with solid endorsement of well-designed rational innovation and with much greater 
financial support for systemic change processes. Doing so will provide new 
opportunities to improve the science-base (assuming evaluation studies are sufficiently 
underwritten). And doing so is essential to moving forward in transformative ways. 
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As an introduction to the literature on implementation and systemic change, our Center 
continues to add to its series of “Information Resources” on enabling systemic change. The 
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> Systemic Change, and the Diffusion of Innovation in Schools - 
http://smhp.psych.ucla.edu/qf/systemicchange.html 
>Change Agent/Organizational Facilitator - 

http://smhp.psych.ucla.edu/qf/changeagent.htm 
>Empiric ally -Supported/Evidence-Based Interventions for Children ’s MH - 
http://smhp.psych.ucla.edu/qf/ests.htm 


11 



Appendix A 


Benchmark Checklist for Monitoring and Reviewing Progress 

in Developing a Comprehensive System to 
Address Barriers to Learning and Teaching 


The checklist on the following pages is designed to aid those involved in the process of 
restructuring education support programs and developing a Learning Supports (Enabling) 
Component. 

This tool was developed as a formative evaluation instrument for use by Steering Groups, 
Organization Facilitators, and other change agents. It aids in focusing problem solving 
discussions and planning next steps. 


The items should be modified to fit local strategic and action plans 


£ UCLA %. 
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Website: http://smhp.psych.ucla.edu 
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Site Name: 

Date 

started 

Date 

Completed 
if applies 

Current Status 

I. ORIENTATION AND CREATING READINESS 

DISTRICT LEVEL— District Name: 




A. Establishment of a district Steering Group 
(“champions”) to facilitate development of the 

Component 

Steering Group members identified 
Names: Position: 




B. Leadership and systemic change training for 
steering group members 




C. Orienting district stakeholders - initial contacts 
made with key stakeholders to introduce basic ideas 

(1) “Social marketing” strategies used to introduce basic 
ideas and relevant research base to key stakeholders 
administrators 
>staff 

>parent representatives 

>business and community stakeholders 

> 




(2) Opportunities for interchange provided & 

additional in-depth presentations made to build 
a critical mass of consensus for systemic changes 




(3) Ongoing evaluation of interest indicates a 
critical mass of stakeholders are ready to 
pursue a policy commitment and/or continuing work 
to enhance commitment for moving forward 




(4) Ratification and sponsorship elicited from 
a critical mass of stakeholders 




D. Establishing Policy Commitment & Framework - 

(follow-up meetings with district leaders to 
clarify dimensions of the work and how to proceed) 

(1) Negotiation of policy commitment and conditions 
for engagement (e.g., high level policy established and 
assurance of leadership commitment - learning 
supports component adopted as one of the primary 
and essential components of school improvement - on 
a par with instructional & management components) 




(2) Policy translated into an inspiring vision, a 
framework, and a strategic plan that phases in 
changes using a realistic time line 
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Current Status 


Date Date 

Site Name: started Completed 

if applies 

(3) Policy translated into appropriate resource 
allocations (leadership, staff, space, budget, time) 

(4) Incentives for change established 

(e.g., intrinsically valued outcomes, expectations 
for success, recognitions, rewards) 

(5) Procedural options established that reflect stakeholder 
strengths and from which those expected to implement 
change can select strategies they see as workable 

(6) Infrastructure and processes established for facilitating 
change efforts 

(7) Establishment of a change agent(s) position 


(8) Change agent(s) identified - indicate name(s): 



(9) Initial capacity-building - essential skills developed 
among stakeholders to begin implementation 


(10) Benchmarks used to provide feedback on progress and 
to make necessary improvements in the process for 
creating readiness 

E. Development of phase-in plan for District (see 
and adapt Start-up and Phase-in tasks for schools ) 

SCHOOL LEVEL- School Name: 

A. Establishment of Steering Group (“champions”) 
for school to facilitate development of Component 
Steering Group members identified 
Names: Position: 


B. Leadership and systemic change training for 

Steering Group members 

C. Orienting school stakeholders - initial contacts 
made with key stakeholders to introduce basic ideas 

(1) “Social marketing” strategies used to introduce basic 
ideas and relevant research base to key stakeholders 
administrators 
>staff 

>parent representatives 

>business and community stakeholders 
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Current Status 


Date Date 

Site Name: started Completed 

if applies 

(2) Opportunities for interchange provided & additional 
in-depth presentations made to build a critical mass of 
consensus for systemic changes 

(3) Ongoing evaluation of interest indicates a critical 
mass of stakeholders are ready to move forward 

(4) Ratification and sponsorship elicited from 
a critical mass of stakeholders 

D. Establishing Policy Commitment & Framework - 

(follow-up meetings with school leaders to 
clarify the dimensions of the work and how to 
proceed) 

(1) Negotiation of a policy commitment and conditions for 
engagement (e.g., high level policy established and 
assurance of leadership commitment - learning 
supports component adopted as one of the primary and 
essential components of school improvement - on a par 

_ _ _w idi dic_in struct ion al_ and m anage UTcnJjcomppncnts) 

(2) Policy translated into an inspiring vision, frameworks, 
and a strategic plan to phase-in changes using a 
realistic time line 

(3) Policy translated into appropriate resource 
allocations (leadership, staff, space, budget, time) 

(4) Incentives for change established 

(e.g., intrinsically valued outcomes, expectations 
for success, recognitions, rewards) 

(5) Procedural options established that reflect stakeholder 
strengths and from which those expected to implement 
change can select strategies they see as workable 

(6) Infrastructure and processes established for facilitating 
change efforts 

(7) Establishment of a change agent(s) position 


(8) Current change agent(s) - indicate name(s): 



(9) Initial capacity-building - essential skills developed 
among stakeholders to begin implementation 


(10) Benchmarks used to provide feedback on progress and 
to make necessary improvements in the process for 
creating motivational readiness and capability to 
move forward 
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Current Status 


Date 

started 


II. START-UP AND PHASE-IN 


A. Evaluation indicating that Steering Group at the 
school is ready to facilitate development of the 

Component 

Update Members 
Name: Position: 


B. Identification of a site leader (equivalent to the 
leader for the Instructional Component) 

Name: Position: 


C. Identification of others advocates for the 
Component 

Names: Position: 


D. Current Change Agent(s) indicate name(s): 



E. Leadership and systemic change training for all 
taking a lead in developing the component 


F. Survey of administrator, teacher, and other staff 
regarding attitudes about new directions 

G. ESTABLISHMENT OF LEARNING 

_ SUPPORTS RESOURCE TEAM 

(1) Identification of potential team members 

(2) Recruitment of team members. 

Name: Position: 


(3) Initial team meeting. 

(4) Training for team. 


Date 

Completed 
if applies 


Site Name: 

E. Development of phase-in plan for school (based 
on the following Start-up and Phase-in tasks ) 
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Site Name: 


Date 

started 


H. INITIAL MAPPING AND ANALYSIS OF 

EXISTING RESOURCES 

(1) Initial mapping 

(2) Initial analyses (e.g., of needs, gaps, efficacy, 
coordination, integration with school improvement 
planning) 

(3) Initial plans and steps to improve learning supports 
(enabling) activity (e.g., priorities, intervention 
development work groups) 

(4) Initial “maps” and plans distributed 

I. INITIAL ENHANCEMENT OF SYSTEMS AND 

ACTIVITY RELATED TO ENABLING 

(1) Analysis, improvement, documentation, and circulation 
of info and recommendations on how to use curie tit 
“systems” - clarification of steps, development of flow 
charts, written descriptions, training of personnel, etc. 
(e.g., for work related to 
>Promoting Healthy Development and 
Preventing Problems 
>Response to Intervention (Rtl) 

>Handling Behavior Problems 

>Referral for Emergency Help-Major Services 

>Triage 

>Care Management 

>Crisis Response (e.g.. Crisis Team) 

> 


(2) Training for existing work groups. 

>Student and Family Assistance Team 
(e.g.. Student Study or Guidance Team) 
>IEP Team members 
>Crisis Team 
>Other (specify) 


J. REFINING INFRASTRUCTURE & 
PURSUING DEEPER MAPPING 
AND ANALYSES 

(1) Learning support activity organized into a delineated 
set of intervention arenas (e.g., six content arenas) 

(2) Standing work groups developed for each arena 

















Current Status 


Date Date 

Site Name: started Completed 

if applies 

(3) Training of Arena work groups 
Specify Areas: 


(4) Initial mapping and analyses of resources related to 
each arena accomplished 

(5) Each arena work group formulates priorities for 
enhancing activity in own area. 

Delineate by Arena: 


(6) Priorities for enhancing learning supports delineated, 
evaluated, and ranked by Learning Supports Leadership 
Team and plans formulated for pursuing top priorities. 

(7) School infrastructure refined so that learning supports 
(enabling) component is fully integrated with the 
instructional and management components 

(8) If relevant, plans formulated to establish a Lamily 
and/or Parent Center 


K. COMPONENT VISIBILITY, 
COMMUNICATION, 

& PROBLEM SOLVING 

(1) Ad hoc work groups developed to enhance component 
visibility, communication, sharing, & problem solving 

(2) All existing programs, services, and resources listed, 
circulated (e.g., to all staff, parents), ), and for high 
visibility featured in memos, bulletins, on websites, in 
a brochure describing the Learning Supports 
Component and key interventions, in newsletters 

and on information boards, etc. 

(3) Other steps taken to enhance visibility. 

(specify) 

(4) Effective communication mechanisms in operation 

(5) Effective problem solving mechanisms in operation 

(6) Effective social marketing mechanisms in operation 
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Site Name 


L. OUTREACH TO FILL GAPS & 
PURSUE ECONOMIES OF SCALE 


Date 

Started 



Current Status 


(1) Formal collaborative linkages established with other 
resources in the district 
(specify) 


(2) Formal collaborative linkages (e.g., a Learning Supports 
Leadership Council) established with other schools in 
locale (e.g., a feeder pattern) 

(specify) 


(3) Formal collaborative linkages (e.g., a school-community 
collaborative) established with a wide range of 
community resources (e.g., programs and agencies) 
(specify) 



M. SYSTEM FOR QUALITY IMPROVEMENT 


Decisions about indicators to be used. 


Members recruited for Quality Improvement Team. 
Name: Position: 


Training of Quality Improvement Team 


Initial Quality Improvement recommendations. 
Made. 

Acted upon. 
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Site Name 

Date 

Started 

Date 

Completed 

Current Status 

III. INSTITUTIONALIZATION 
( maintenance & evolution ) 

& IV. PLANS FOR ONGOING RENEWAL 

A. Indications of planning for maintenance 

(1) policy commitments 




(2) regular budget allocations 




(3) ongoing administrative leadership 




(4) a key facet of school improvement plans 




B. Strategies in use for maintaining 
momentum/progress. (sustainability) 
(List most prominent examples) 




C. Strategies in use and future plans for generating 
renewal 

(List most prominent examples) 





An overarching benchmark involves the monitoring 
of the implementation of evaluation plans. 
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Appendix B 


Examples of Indicators of Impact 


Students 

Increased knowledge, skills, & attitudes related to academics 

• assessed in keeping with curricula standards 
Increased knowledge, skills, & attitudes for enhancing 

• Acceptance of responsibility (including attending, following directions and agreed upon rules/laws ) 

• Self-esteem & integrity 

• Social and working relationships 

• Self-evaluation and self-direction/regulation 

• Physical functioning 

• Health maintenance 

• Safe behavior 

Reduced barriers to school attendance and functioning by addressing problems related to 

• Health 

• Lack of adequate clothing 

• Dysfunctional families 

• Lack of home support for student improvement 

• Physical/sexual abuse 

• Substance abuse 

• Gang involvement 

• Pregnant/parenting minors 

• Dropouts 

• Need for compensatory learning strategies 
Families & Communities 

• Increased social and emotional support for families 

• Increased family access to special assistance 

• Increased family ability to reduce child risk factors that can be barriers to learning 

• Increased bilingual ability and literacy of parents 

• Increased family ability to support schooling 

• Increased positive attitudes about schooling 

• Increased home (family/parent) participation at school 

• Enhance positive attitudes toward school and community 

• Increased community participation in school activities 

• Increased perception of the school as a hub of community activities 

• Increased partnerships designed to enhance education & service availability in community 

• Enhanced coordination & collaboration between community agencies and school programs 
& services 

• Enhanced focus on agency outreach to meet family needs 

• Increased psychological sense of community 

Programs & Systems 

• Enhanced processes by which staff and families learn about available programs and services 
and how to access those they need 

• Increased coordination among services and programs 

• Increases in the degree to which staff work collaboratively and programmatically 

• Increased services/programs at school site 

• Increased amounts of school and community collaboration 

• Increases in quality of services and programs because of improved systems for requesting, 
accessing, and managing assistance for students and families (including overcoming 
inappropriate barriers to confidentiality) 

• Establishment of a long-term financial base 
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